Apace2 stops serving pages.

Apace2 stops serving pages.

am 23.09.2009 22:40:26 von Middle Fork GIS

--00504502d4b39180ce047444b908
Content-Type: text/plain; charset=ISO-8859-1

Hi, I am a brand new user to this forum though I've been successfully
managing (low volume) Apache Servers for a number of years now. The reason
I'm here today is that periodically, though apparently (?) at random
intervals, Apache stops serving pages.

Using wget from the server host:

# wget hipcodes.com
--2009-09-23 14:32:15-- http://hipcodes.com/
Resolving hipcodes.com... 72.249.77.11
Connecting to hipcodes.com|72.249.77.11|:80... connected.
HTTP request sent, awaiting response...

And there it sits until the server is started; likewise from a remote
browser, the page is never delivered. Nothing appears in Apache's log,
even when set to the most verbose 'INFO' level.

Apache is NOT pegging the CPU: CPU usage will show close to 0% and the load
average will show close to zero as well. Apache is not hogging the memory,
very little memory usage is shown.

Apache2.2.13-1
VPS running: Debian: 2.6.18-028stab062.3 #1 SMP Thu Mar 26 14:46:38 MSK
2009 i686 GNU/Linux

Traffic is around 10,000 hits/day, a combination of static files, PHP, AJAX,
from an interactive mapping program named Ka-Map! (similar to G**gle Maps,
et al), with queries against a PostgreSQL/PostGIS database backend.

I had the (I think) exact same configuration running on a similar VPS for
over a year and this never happened, as far as I can tell.

What I have done so far (all to no effect):

enable/disable mpm_prefork
increase/decrease MaxSpareServers as high as 250
enable/disable KeepAlive
disable as many additional modules as possible.


The apache2.conf currently looks like this:

ServerRoot "/etc/apache2"
LockFile /var/lock/apache2/accept.lock
PidFile ${APACHE_PID_FILE}
Timeout 300
KeepAlive Off
MaxKeepAliveRequests 50
KeepAliveTimeout 2


StartServers 5
MinSpareServers 5
MaxSpareServers 50
MaxClients 250
MaxRequestsPerChild 0



StartServers 10
MinSpareThreads 25
MaxSpareThreads 75
ThreadLimit 64
ThreadsPerChild 25
MaxClients 250
MaxRequestsPerChild 0



StartServers 10
MaxClients 250
MinSpareThreads 25
MaxSpareThreads 75
ThreadLimit 64
ThreadsPerChild 25
MaxRequestsPerChild 0


User ${APACHE_RUN_USER}
Group ${APACHE_RUN_GROUP}
AccessFileName .htaccess

Order allow,deny
Deny from all


DefaultType text/plain
HostnameLookups Off
ErrorLog /var/log/apache2/error.log
LogLevel debug
Include /etc/apache2/mods-enabled/*.load
Include /etc/apache2/mods-enabled/*.conf
Include /etc/apache2/httpd.conf
Include /etc/apache2/ports.conf
LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\"
\"%{User-Agent}i\"" vhost_combined
LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\""
combined
LogFormat "%h %l %u %t \"%r\" %>s %O" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent
CustomLog /var/log/apache2/other_vhosts_access.log vhost_combined
Include /etc/apache2/conf.d/
Include /etc/apache2/sites-enabled/


More information needed?
Any tips?
Tests I can perform?

Thanks in advance.

Steve Walker
middleforkgis aht gmail.com

--00504502d4b39180ce047444b908
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi, I am a brand new user to this forum though I've been successfu=
lly managing (low volume) Apache Servers for a number of years now. =A0The =
reason I'm here today is that periodically, though apparently (?) at ra=
ndom intervals, Apache stops serving pages.




Using wget from the server host:

iv># wget om" target=3D"_blank">hipcodes.com
--2009-09-23 14:32:15-- =3D"http://hipcodes.com/" target=3D"_blank">http://hipcodes.com/



Resolving hipcodes.com... 72.249.77.11
Connecting to ipcodes.com" target=3D"_blank">hipcodes.com|72.249.77.11|:80... connect=
ed.
HTTP request sent, awaiting response...

ace=3D"courier new,monospace">



And there it sits until the server is started; likewise f=
rom a remote browser, the page is never delivered. =A0 Nothing appears in A=
pache's log, even when set to the most verbose 'INFO' level. iv>



Apache is NOT pegging the CPU: =A0CPU usage will show c=
lose to 0% and the load average will show close to zero as well. =A0 Apache=
is not hogging the memory, very little memory usage is shown. =A0
v>


Apache2.2.13-1
t>
VPS running: =A0Debian: =A02.6.18-028stab062.3 #1 SMP Thu Mar =
26 14:46:38 MSK 2009 i686 GNU/Linux

Traffic is aro=
und 10,000 hits/day, a combination of static files, PHP, AJAX, from an inte=
ractive mapping program named Ka-Map! (similar to G**gle Maps, et al), with=
queries against a PostgreSQL/PostGIS database backend.




I =A0had the (I think) exact same configuration running=
on a similar VPS for over a year and this never happened, as far as I can =
tell. =A0

What I have done so far (all to no e=
ffect):



enable/disable mpm_prefork
increase/decrease =
MaxSpareServers as high as 250
enable/disable KeepAlive
>disable as many additional modules as possible.




The apache2.conf currently looks like this:
r>
ServerRoot "/etc/apache2"
LockFile /var/lock/apac=
he2/accept.lock
PidFile ${APACHE_PID_FILE}
Timeout 300
KeepAlive O=
ff


MaxKeepAliveRequests 50
KeepAliveTimeout 2

<IfModule mpm_prefo=
rk_module>
StartServers 5
MinSpareServers 5=

MaxSpareServers 50
MaxClients 250
MaxRe=
questsPerChild 0


</IfModule>

<IfModule mpm_worker_module>
StartSer=
vers 10
MinSpareThreads 25
MaxSpareThreads =
75
ThreadLimit 64
ThreadsPerChild 25
Ma=
xClients 250


MaxRequestsPerChild 0
</IfModule>

<IfModule mpm_ev=
ent_module>
StartServers 10
MaxClients 25=
0
MinSpareThreads 25
MaxSpareThreads 75
Thr=
eadLimit 64


ThreadsPerChild 25
MaxRequestsPerChild 0
</IfModule=
>

User ${APACHE_RUN_USER}
Group ${APACHE_RUN_GROUP}
AccessF=
ileName .htaccess
<Files ~ "^\.ht">
Order allow,d=
eny


Deny from all
</Files>

DefaultType text/plain
Hostna=
meLookups Off
ErrorLog /var/log/apache2/error.log
LogLevel debug
I=
nclude /etc/apache2/mods-enabled/*.load
Include /etc/apache2/mods-enable=
d/*.conf


Include /etc/apache2/httpd.conf
Include /etc/apache2/ports.conf
LogFo=
rmat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\=
" \"%{User-Agent}i\"" vhost_combined
LogFormat "=
;%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{=
User-Agent}i\"" combined


LogFormat "%h %l %u %t \"%r\" %>s %O" common
LogF=
ormat "%{Referer}i -> %U" referer
LogFormat "%{User-ag=
ent}i" agent
CustomLog /var/log/apache2/other_vhosts_access.log vho=
st_combined


Include /etc/apache2/conf.d/
Include /etc/apache2/sites-enabled/

=

More information needed?
Any tips? >
Tests I can perform?

Thanks in advance. >


Steve Walker

>





--00504502d4b39180ce047444b908--

Re: Apace2 stops serving pages.

am 24.09.2009 02:13:48 von Nicholas Sherlock

Middle Fork GIS wrote:
> [...] at random intervals, Apache stops serving pages.
>
> HTTP request sent, awaiting response...
>
> And there it sits until the server is started; likewise from a remote
> browser, the page is never delivered. Nothing appears in Apache's log,
> even when set to the most verbose 'INFO' level.

The log is only written once the page has been served, so that isn't
surprising.

> Apache is NOT pegging the CPU: CPU usage will show close to 0% and the
> load average will show close to zero as well. Apache is not hogging
> the memory, very little memory usage is shown.

I'd take a look at your database. If one query locks all the important
tables, then runs for minutes and minutes, every other query will block
waiting for it. This will quickly fill up your Apache connection limit
with people waiting for their response to be delivered. Eventually, the
bad query times out and everything goes back to normal.

On MySQL, this would be diagnosed with a quick "SHOW PROCESSLIST", I'm
not sure about PostgreSQL though.

Cheers,
Nicholas Sherlock


------------------------------------------------------------ ---------
The official User-To-User support forum of the Apache HTTP Server Project.
See for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: Re: Apace2 stops serving pages.

am 24.09.2009 15:24:47 von Middle Fork GIS

--00504502d4b35ef24e047452c1db
Content-Type: text/plain; charset=ISO-8859-1

Argh it happened again overnite.

I checked in first thing this morning and Apache had been hung for 8
bleeding hours. Zero pages served in all that time. Postgresql did not
seem to be at fault. my query against it worked fine. Also, all of my
queries are SELECT, so they are non-exclusive and shouldn't result in any
locks.

Also, when it hangs, NO requests get through, even those to static html
files.

I did note that all 250 of the spawned apache processes were still alive.




On Wed, Sep 23, 2009 at 5:13 PM, Nicholas Sherlock wrote:

> Middle Fork GIS wrote:
>
>> [...] at random intervals, Apache stops serving pages.
>>
>> HTTP request sent, awaiting response...
>>
>> And there it sits until the server is started; likewise from a remote
>> browser, the page is never delivered. Nothing appears in Apache's log,
>> even when set to the most verbose 'INFO' level.
>>
>
> The log is only written once the page has been served, so that isn't
> surprising.
>
> Apache is NOT pegging the CPU: CPU usage will show close to 0% and the
>> load average will show close to zero as well. Apache is not hogging the
>> memory, very little memory usage is shown.
>>
>
> I'd take a look at your database. If one query locks all the important
> tables, then runs for minutes and minutes, every other query will block
> waiting for it. This will quickly fill up your Apache connection limit with
> people waiting for their response to be delivered. Eventually, the bad query
> times out and everything goes back to normal.
>
> On MySQL, this would be diagnosed with a quick "SHOW PROCESSLIST", I'm not
> sure about PostgreSQL though.
>
> Cheers,
> Nicholas Sherlock
>
>
> ------------------------------------------------------------ ---------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>

--00504502d4b35ef24e047452c1db
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Argh it happened again overnite.

I checked in=
first thing this morning and Apache had been hung for 8 bleeding hours. =
=A0Zero pages served in all that time. =A0Postgresql did not seem to be at =
fault. =A0my query against it worked fine. =A0Also, all of my queries are S=
ELECT, so they are non-exclusive and shouldn't result in any locks. v>

Also, when it hangs, NO requests get through, even thos=
e to static html files.

I did note that all 250 of=
the spawned apache processes were still alive.=A0

>



On Wed, =
Sep 23, 2009 at 5:13 PM, Nicholas Sherlock < "mailto:n.sherlock@gmail.com" target=3D"_blank">n.sherlock@gmail.com>=
;
wrote:


x #ccc solid;padding-left:1ex">Middle Fork GIS wrote:

x #ccc solid;padding-left:1ex">
[...] at random intervals, Apache stops serving pages.




HTTP request sent, awaiting response...



And there it sits until the server is started; likewise from a remote brows=
er, the page is never delivered. =A0 Nothing appears in Apache's log, e=
ven when set to the most verbose 'INFO' level.




The log is only written once the page has been served, so that isn't su=
rprising.




x #ccc solid;padding-left:1ex">
Apache is NOT pegging the CPU: =A0CPU usage will show close to 0% and the l=
oad average will show close to zero as well. =A0 Apache is not hogging the =
memory, very little memory usage is shown. =A0




I'd take a look at your database. If one query locks all the important =
tables, then runs for minutes and minutes, every other query will block wai=
ting for it. This will quickly fill up your Apache connection limit with pe=
ople waiting for their response to be delivered. Eventually, the bad query =
times out and everything goes back to normal.





On MySQL, this would be diagnosed with a quick "SHOW PROCESSLIST"=
, I'm not sure about PostgreSQL though.



Cheers,

Nicholas Sherlock





------------------------------------------------------------ ---------

The official User-To-User support forum of the Apache HTTP Server Project.<=
br>
See <URL: lank">http://httpd.apache.org/userslist.html> for more info.

To unsubscribe, e-mail: g" target=3D"_blank">users-unsubscribe@httpd.apache.org

=A0" =A0 from the digest: httpd.apache.org" target=3D"_blank">users-digest-unsubscribe@httpd.apache.o=
rg


For additional commands, e-mail: org" target=3D"_blank">users-help@httpd.apache.org






--00504502d4b35ef24e047452c1db--